Digital cameras, and other digital imaging devices, are becoming increasingly common both in the field of still photography, as well as in the field of motion imaging as is evidenced by the proliferation of digital cameras and video recorders. Digital imaging has the advantage over its counterparts, which utilize conventional media such as silver halide film, that the results are instantly available for viewing, editing, printing, and other forms of utilization.
Color digital imaging devices generally utilize an array of color sensors to capture the digital image information. The sensors typically have spectral sensitivities that correspond roughly to the red, green and blue portions of the visible spectrum. Alternatively, sensors with other spectral sensitivities such as cyan, magenta and yellow, or cyan, magenta, yellow and green have also been used. Typically, the exposure values captured by the color sensors are integerized to form quantized color values (code values) and stored in memory for later processing. The later processing blocks typically include color correction, as well as a variety of spatial processing blocks such as sharpening and/or noise removal.
Digital imaging devices are typically designed so that a nominal gray patch in the subject area of the image should be encoded with specified code values. For digital cameras, some type of exposure meter is typically used to determine a lens aperture and exposure time setting that is appropriate for the level of scene illumination. When an electronic flash is used, the flash exposure level is also typically controlled using some form of exposure meter. Other types of digital imaging devices may utilize other forms of exposure control.
Exposure errors can cause the nominal gray patch to come out at code values that are lower or higher than expected. Typical causes of exposure errors are scenes that are dominated by dark or light objects, and scenes where the subject and the background have significantly different exposure levels. For example, back-lit scenes where the background is brighter than the subject, and flash-lit scenes where the background is at a much larger distance than the subject. In some cases, such as a digital scanner designed to scan photographic prints or negatives, the exposure errors may be inherent in the "scene" itself. For example, if a slide scanner may be used to scan an overexposed slide.
In addition to exposure level errors, color balance errors can also be introduced by variations in the spectral content of the scene illumination. Typically, digital imaging devices are designed so that equal color signals will be generated for neutral scene objects. Color balance errors cause the relative proportions of the color signals for a neutral scene object to deviate from the expected value. For example, if a digital camera is calibrated so that equal red, green, and blue color signals are obtained when a neutral object is photographed with typical daylight illumination, non-equal color signals will generally result when the neutral object is photographed under tungsten illumination.
These exposure level and color balance problems are analogous to those that occur in traditional silver halide photography. In that case, "scene balance algorithms" have been developed to analyze a photographic negative and adjust a set of color filter values and/or exposure times in an optical printing system to compensate for the exposure and color balance errors. (For example, see "Automatic Color Printing Techniques," Image Technology, April/May 1969, pp. 39-43; "Tone Correction Apparatus for Color Analyzers," Eastman Kodak Research Disclosure, Nov. 1974, pp. 24-26; U.S. Pat. No. 4,101,217 assigned to AGFA-Gevaert A. G.; U.S. Pat. No. 4,707,119 and 4,984,013 assigned to Fuji Photo Film Co., Ltd.; and U.S. Pat. No. 5,016,043 assigned to Gretag Systems.)
Scene balance algorithms typically work by analyzing the distribution of overall exposure levels and relative color values in an image to determine the appropriate level of exposure and color balance compensation that is needed. Frequently, these algorithms work by first computing a low-resolution version of the image and then analyzing that image. The low-resolution image is sometimes referred to as a "paxelized" image. The term "paxel" as used throughout this specification shall mean a color value determined by averaging the color values for a group of pixels.
Scene balance algorithms can also be used for digital imaging devices. In this case, the scene balance algorithms must typically be tuned to the color response characteristics of a particular digital imaging device. For example, different digital film scanners will have different sensor spectral sensitivities. Additionally, different film scanners may also use different code value vs. density quantization functions. Therefore, the distribution of code values associated with a particular image will be a function of the digital imaging device that is used to capture the image. Since most scene balance algorithms rely on an analysis of the distribution of the image code values to determine the appropriate corrections, this would necessitate developing different versions of the algorithm to work with different digital imaging devices.
A scenario where different versions of the scene balance algorithm are used for different digital imaging devices is shown in FIG. 1. In this case, an image is captured by two different digital imaging devices 10 and 11. Input color values generated by the first digital imaging device 10 are in a first device dependent color space and are shown as R.sub.1, G.sub.1 and B.sub.1. Likewise, input color values generated by the second digital imaging device 11 are in a second device dependent color space and are shown as R.sub.2, G.sub.2 and B.sub.2. A first scene balance algorithm 12 optimized for images in the first color space is shown operating on the first set of input color values to produce balanced color values R.sub.1B, G.sub.1B and B.sub.1B. A second scene balance algorithm 13 optimized for images in the second color space is shown operating on the second set of input color values to produce balanced color values R.sub.2B, G.sub.2B and B.sub.2B. Following the scene balance blocks, it is typically necessary to perform a color correction block in order to produce color signals that are appropriate for display on a particular output device 16, or for storage in a standard image format. Since the color spaces associated with the digital imaging devices are different, it is also necessary to use different color correction transformations for each digital imaging device. These color correction transformations are shown as 14 and 15 for the first and second imaging devices, respectively. Applying the color correction transformations to the balanced color values produces output color values R.sub.O, G.sub.O and B.sub.O that are appropriate for display with the output device 16.
Since developing many variations of the scene balance algorithm for different devices is quite time consuming, an alternate approach can be used which involves first transforming the digital image color values to a standard representation where a fixed scene balance algorithm can be employed. For example, in FIG. 2, the scene balance algorithm is applied following the color correction transformations. In this case, an image is captured by two different digital imaging devices 20 and 21. Input color values generated by the first digital imaging device 20 are in a first device dependent color space and are shown as R.sub.1, G.sub.1 and B.sub.1. Likewise, input color values generated by the second digital imaging device 21 are in a second device dependent color space and are shown as R.sub.2, G.sub.2 and B.sub.2. Color correction transformations 22 and 23 are applied to the color values for the first and second imaging devices to produce output color values R.sub.O, G.sub.O and B.sub.O in an output color space that is appropriate for display with the output device 25. A scene balance algorithm 24 optimized for images in the output color space is shown operating on the output color values to produce balanced output color values R.sub.OB, G.sub.OB and B.sub.OB.
The arrangement shown in FIG. 2 has an advantage over that shown in FIG. 1 in that only a single scene balance algorithm needs to be developed and optimized. However, there is also an important disadvantage associated with this technique. Since the output device has a finite dynamic range and color gamut, a large amount of image information is generally lost in the process of applying the color correction transformations to the digital imaging device color values. In particular, digital imaging devices are typically designed so that the dynamic range associated with the input color values is large enough to encode image intensity values that are substantially higher than those that would be associated with an output device. This extra dynamic range can be as large as an extra factor of 2 or more. The color correction transformation will then apply a set of tone/color reproduction aims to compress the extra dynamic range and color gamut associated with the input color values into the displayable gamut of the output device. An important part of the color correction transformation is a tonescale function that relates the input intensity values to the output device intensity values. A typical tonescale curve 30 that can be used is shown in FIG. 3. The horizontal axis represents the logarithm base 10 of the input exposure value. A log exposure of 0.0 corresponds to the exposure obtained from objects in the image that have a reflectance of 1.0. The vertical axis represents the negative of the logarithm of the image intensity. This is analogous to the optical "density" of a photographic print. It can be seen that the input exposure values outside a range of -2.0 to 0.0 are compressed into small ranges of image intensities. As a result, much of the image information that was captured in this range is lost during the application of the color correction transformation. This loss of image information is particularly important if the scene balance algorithm determines that the image was overexposed. In the case, it is necessary to darken the image accordingly. However, much of the information that could be used to darken the image corresponds to that which was lost in the color correction transformation. Therefore, the correction introduced by the scene balance algorithm will be less optimal than that associated with the method of FIG. 1 where the images were corrected in the input color space.
A third configuration is shown in FIG. 4 that overcomes many of the disadvantages associated with the arrangement of FIG. 2. In this case, an image is captured by two different digital imaging devices 40 and 41. Input color values generated by the first digital imaging device 40 are in a first device dependent color space and are shown as R.sub.1, G.sub.1 and B.sub.1. Likewise, input color values generated by the second digital imaging device 41 are in a second device dependent color space and are shown as R.sub.2, G.sub.2 and B.sub.2. Color transformation blocks 42 and 43 are applied to the color values for the first and second imaging devices to produce standard color space values R.sub.S, G.sub.S and B.sub.S in an standard color space. The standard color space being chosen so that the extended dynamic range and color gamut associated with the input color spaces can be encoded without significant loss of information. A scene balance algorithm 44 optimized for images in the standard color space is shown operating on the standard color space values to produce balanced standard color space values R.sub.SB, G.sub.SB and B.sub.SB. Since the standard color space values are not appropriate for display on an output device 46, a color correction transformation 45 is then used to compute output color values R.sub.O, G.sub.O and B.sub.O.
Although the method of FIG. 4 overcomes many of the limitations associated with FIGS. 1 and 2, there is also a disadvantage to this approach. It can be seen that two color transformation operations are now necessary to process the image. One color transformation to determine the standard color space values from the input color space values, and a second color transformation to determine the output color values from the balanced standard color space values. Therefore, the computation time and complexity associated with the color processing operations is increased significantly relative to the methods shown in FIGS. 1 and 2.